Method and system for 3-D object modeling

ABSTRACT

3-D views of a 2-D image containing objects are provided by simulation of inter-object volume and intra-object volume. Intra-object volume is simulated by defining a plurality of volumetric slices from the object and assigning relative volume values to each. Multiple views of the object are simulated by displacing each slice in accordance with the defined volumetric effect as part of an animation sequence simulating progressive movement about a focus point of the object.

CROSS-REFERENCE TO RELATED APPLICATION

This application is a continuation of and claims priority from U.S.patent application Ser. No. 10/211,500, entitled “Method and System for3-D Object Modeling” filed Aug. 2, 2002, now pending, which isincorporated herein by reference.

FIELD OF THE INVENTION

The present invention relates to graphical modeling, specifically theinvention relates to volumetric simulation from a two dimensional inputimage.

BACKGROUND OF THE INVENTION

Stereoscopic systems present a viewer with an image of an object suchthat the object appears to have depth. The wide variety of techniques toprovide such images commonly involve presenting different views of theobject to each eye to simulate such images. Traditional stereoscopicsystems employ an apparatus that keeps the left and right eye imagesdirected solely at the appropriate eye such as by employing helmets orshutter glasses.

Autostereoscopic displays do not require special viewing equipment. Suchdisplays are attractive because they offer the closest approximation tohow people view the “real” world around us, unencumbered by externalviewing apparatus. Lenticular surfaces, used to simulate a 3-D image ona 2-D surface, are an example of autostereoscopic displays. The imagepresented on the lenticular surface typically includes at least oneobject and a background. Each object is associated with a relativespacing from the background, and in some cases, the other objects, so asto provide a view of the image as would appear to a moving observer of a3-D scene. However, the present techniques for providing 3-D lenticularimages are limited in that they only provide for convenient simulationof depth between objects and the background, and between the objectsthemselves. Moreover, the present techniques are not easily automatedby, for example, a computer program. Furthermore, present techniques mayonly provide an adequate simulation of depth when the angle of view tothe lenticular surface is changed. Accordingly, there is a need for amethod for providing an image on a lenticular surface which simulatesboth intra- object volume and inter-object volume. There is a furtherneed for a method of inter-object volume simulation which is adapted forautomation.

SUMMARY OF THE INVENTION

Therefore, in accordance with the present invention, a series of framesare provided as part of a lenticular image. Each frame provides ananimation state with certain relative displacement between objects ofthe image. Furthermore, each object of the image is subdivided into aplurality of layers in accordance with a mask series. The layers aredisplaced relative to one another in accordance relative orientation forvarious observation points, each corresponding to a frame in a framesequence. Accordingly, the series of frames provides simulation ofobject volume by relative displacement of layers within the object. Theseries of frames also provides an animation sequence that includesvarious angle views of the object.

In one embodiment the invention provides a method for producing an inputimage for application from a two dimensional image onto an image surfaceadapted to present interlaced frames of the input image. The methodreceives an input image, which includes at least one object. The methodreceives layer definitions for a plurality of layers from at least oneobject of the input image. The method receives an indication of thetotal displacement for the object animation. The method then determinesa relative displacement for each layer from the plurality of layers. Themethod generates a sequence of image frames by progressively offsettingsaid plurality of layers in accordance with relative layer displacementand a total animation displacement. Finally, the method interlaces theanimated sequence of frames to a combined image to provide an inputinterlaced image for application to said image surface.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 illustrates a typical lenticular sheet;

FIG. 2 illustrates a plurality of image frames and an interlaced image;

FIG. 3 is a flow diagram illustrating a method for simulating a 3-Dimage of a 2-D source image in accordance with the invention;

FIG. 4 illustrates a method for defining a mask series for an object ofan input image;

FIG. 5A illustrates an input image with an overlaid grid;

FIG. 5B illustrates an object of the input image;

FIG. 5C illustrates a first layer of the object;

FIG. 5D illustrates a second layer of the object;

FIG. 5E illustrates a third layer of the object;

FIG. 5F illustrates a fourth layer of the object; and

FIG. 6 illustrates a frame sequence for the object.

DETAILED DESCRIPTION OF THE INVENTION

FIG. 1 illustrates a lenticular surface and a corresponding interlacedimage 32. “Lenticule” is a synonym for “lens” but has come to mean asheet of long thin lenses. The Lenticular surface preferably contains aseries of cylindrical lenses 30 molded into a plastic substrate. Eachlens focuses on a strip from an image on the back side of the lenticularsurface 32. As illustrated in FIG. 1, the lenticular image is positionedso that the line of sight for each one of an observer's eyes 20, 21 isfocused onto different groups of strips 24, 26, 28 and 25, 27, 29,associated with different images. The lenticular sheet is preferablymade so that the back side of the sheet 32 is exactly one focal lengthbehind the lens 30 so that the image data emerges collimated from eachindividual lens.

FIG. 2 illustrates a common arrangement of frame portions in aninterlaced image. The illustrated case is for N sub-images 34 that areinterlaced to form a composite 36 that is N times wider than theindividual images. The final image 38 is stretched vertically so as toform a lenticular image than is of the same proportions as thesub-images 34 but N time larger (vertically a well as horizontally). Itshould be noted that the final stretching could be performed just beforeprinting. Hence, the vertically stretched version need not be saved.

A number of method are employed in the prior art to create the Nsub-images when 3-D information is available. For example, if a computerbased 3-D model and rendering package are available, the N sub-imagesmay be created by positioning of the simulated camera at various anglesby lateral movement around the object. The same is with a physicalstatic object that is available, where positioning of the camera is theonly requirement. Using a computer model, a focal point is usuallychosen in the middle of the object of interest. The computer cameraposition is then moved in regular steps to vary the viewing angle to thefocal point. As may be appreciated, the exact step size and the numberof images depends on the lens spacing of the lenticular sheets beingused, the resolution of the printing system available, and the qualityof the stereopis.

Often times, it is not possible to move a camera position about asubject, either by computer program or by physical relocation. Forexample, the object may be from a photograph taken some time before thesimulation effort and is no longer available for acquiring multiple viewimages. At other times, the photographed subject is in motion or iscontinuously changing such as a splashing drink, where capturingmultiple angles of the object is expensive and difficult. Furthermore, ascene may require extensive manipulation and the addition of numerousobjects or may include a hand drawing for which depth information is notavailable. Accordingly, the present invention provides a method forsimulating 3-D views of an object without acquiring images of the objectat multiple angles or employing computer based 3-D model and 3-Drendering package. Rather, the invention employs a mask series andmotion definitions specific to the objects to provide animated frames ofthe object. These animated frames are then incorporated into aninterlaced image for the lenticular surface or are provided to a motionsequence viewer as an animation sequence.

FIG. 3 is an overview of method steps for generating simulated 3-D imageviews at an object from an input 2-D image. The system receives a twodimensional image, which includes at least one object (step 40).Examples of such input images include photographs, art illustrations,and computer generated 2-D images. The image scan quality is preferablyat least 300 dpi, CMYK, 60 MB file size. The scan and color arecorrected and retouched as needed based on the scan result. Furthermore,the border outline of any object which is to be rendered must berecreated as part of the background to allow for the object portions tobe displaced in an animation without creating pixel gaps. At least oneobject of the image is selected for 3-D representation (step 42). Thesystem applies a mask series to at lease one object of the image toprovide a plurality of object layers. The system receives a totaldisplacement indication for the layer animation (step 45). A framesequence is generated by employing the defined layers (46).

The following discussion, with reference to FIGS. 4, 5, and 6,illustrates one method for constructing a mask series for defininglayers for an object in accordance with the invention. As may beappreciated other methods may be employed to define the layers, whichsubdivide an object to a plurality of layers for 3-D simulation, withoutdeparting from the teachings of the invention. The mask series, definingthe layers from the object, is preferably constructed by a designer asdiscussed herein, based on estimations and observation for theparticular object type. The depths of various portions of the object aregenerally not known and therefore are estimated by the designerconstructing the mask series. To estimate such relative depths, in oneembodiment, a perspective is selected from which the object is presentedto the viewer. The perspective includes an angle of view and depth offield. In one embodiment, one or more simulated profile views of theobject are constructed to aid the designer in selecting the relativedepths of various layers. In this embodiment, a plan is outlined toarrange the various co-planer layers, which make up the final 3-Dsimulated image. The different layers are grouped as part of theforeground, middle-ground, or background. The grouping is preferablyused to later facilitate combining simulation frames of the objects,with proper relative depths, as one frame sequence, when the simulationframes for each object are independently generated.

Each object is preferably divided to a plurality of layers, which aredefined by the mask series. The mask series is preferably generated bythe designer outlining and cutting away select portions of the object todefine a plurality of layers. Each layer is selected so as to includeportions of the object that are relatively in the same cross-sectionplane of the object. As may be appreciated, the object portions in alayer may include non-continuous portions which share a common plane. Asillustrated below with reference to FIG. 4, the layers are preferablydefined starting from the portions of the object closest to the selectedobservation point and ending with the portions of the object farthestfrom the observation point. The number of defined layers for the objectdepends on the desired depth resolution and desired volumetric effect ofthe final simulation, the level of desired detail, and the totalmovement for all layers. As may be appreciated, the greater the numberof layers, the better the depth resolution and corresponding volumetriceffect. The mask series preferably defines layers which include portionsof subsequent or previous layers. In one embodiment subsequent layersshare common edges which are set to soft silluette to provide almost atransparent appearance. However, in a preferred embodiment, layers donot share more than a minimum number of pixels so as to avoid pixeloverlap and other repetition during the animation when layers aredisplaced relative to one another.

FIG. 4 illustrates a method for defining a mask series for an object. Asdiscussed above, the edges of the object and of each layer arepreferably set to soft silhouette so as to easily integrate with theremainder of the image and with other layers. The nearest points to theobserver containing portions of the object are located by approximatingthe relative depth of object portions (step 50). The nearest point tothe observer is preferably associated with the least displacement of allpoints in the object to be animated. As may be appreciated, the locationof the nearest point within the object's scenery space can be varied torelocate the object within the scenery while maintaining the volumetricappearance of the object, as provided by the animation. A layered gridis optionally superimposed on the object with its center within theidentified nearest points to aid the designer in defining layers (step52). A first mask boundary is defined to include object portions whichcontain the nearest points (step 54). The object portions within themask boundary are copied as a first layer and are defined as a firstcomponent (step 56). Preferably, each layer is named with acorresponding relative displacement, which is used during the subsequentanimation. The mask boundary is expanded to include additional portionsof the object around each of the previous near portions (step 58). Theboundary shape is preferably adjusted to generally include portionssharing another common plane, further away from the observer. Thecurrent object portions within the new mask edges are copied as a nextcomponent (step 60). The first component is erased from the secondcomponent and the result is saved as a new layer (step 64). The secondcomponent is renamed as the first component (step 66). The resultantlayers are preferably stored as sequential layers. The designerdetermines whether the current component border is outside the objectborder (step 62). As may be appreciated, when the border is expanded,the designer ensures that the mask does not extend outside the objectborder unless all object portions have been copies to layers. If thecomponent border is outside the object border, whereby it is the lastcomponent, the designer proceeds to define a relative displacement foreach layer, as discussed above, by including the displacement value foreach layer as part of the layer's stored data (step 68). Accordingly,the layer definitions provide a series of mask patterns. The definedmask patterns are stored for subsequent application to objects fromother images. After relative displacement is assigned to each layer, thelayers are animated by an animation program to provide for a framesequence, as is known in the art (step 70). In one embodiment, thedisplacement for each layer includes both a relative movement value anda direction (+/−). One example animation program is Adobe AfterEffectfrom Adobe Systems. The animated frame sequence is interlaced forapplication to a lenticular surface (step 72). Alternatively theanimated frame sequence is provided to a motion video player as a moviesequence.

FIGS. 5A-5F illustrate an input image and a series of layer dissectionsfor an object of the image. FIG. 5A illustrates an input image with agrid overlay as discussed with reference to step 52 of FIG. 4. FIG. 5Billustrates part of an object of the image, from which several layersare defined. The defined layers are illustrated in FIGS. 5C-5F. Asdiscussed above, each layer includes portions of the object that are onthe same cross-section plane from the observation point. Accordingly,each plane is a virtual slice of the object, wherein the slice isperpendicular to the observation axis. The arrows in the illustration ofFIG. 5B indicate the nearest points for the object. The first layerdefined from the object is around the defined near points. FIG. 5Cillustrates the first layer of the object. The mask border is expendedbeyond the first layer border, to include additional portions of theobject. The portions of the first layer are removed from the newlydefined area to provide the second layer. FIG. 5D illustrates the secondlayer of the object. The mask border is further expanded beyond thesecond layer border, to include additional portions of the object. Thesecond defined component is erased from the current component to providethe next component, the third layer. FIG. 5E illustrates the thirdlayer. The mask border is further expanded beyond the third layerborder, to include additional portions of the object. The third definedcomponent is erased from the current component to provide the nextcomponent, the fourth layer. FIG. 5F illustrates the fourth layer. Asmay be appreciated, the process continues to provide for a plurality oflayers from the object.

Each frame of the animation sequence, employed to provide the interlacedlenticular image, includes a snapshot of object layers at varioushorizontal displacements, based on the particular observation point andcorresponding focus point. The animation of object layers of step 70 inFIG. 4 is preferably by a sub-process separate from the layer definitionprocess. To generate the frame sequence corresponding to each object,the layers defined in the earlier stage are shifted relative to oneanother in accordance with the frame order in the sequence and theselected focus point. The layer displacement in each frame is preferablyalong a horizontal axis. Depending on the location of the layer relativeto the focus point, the displacement is either in the left direction orin the right direction. For example, when a first layer is on one sideof the focus point and a second layer is on the opposite side of thefocus point, the first layer will be displaced in one direction and thesecond layer will be displaced in an opposite direction so as tosimulate rotation about the focus point.

The relative displacement value for each layer is preferably determinedby reference to the desired total displacement for the object.Specifically, the relative displacement for each layer depends on itsrelative distance from the focus point and the maximum displacement forthe layer farthest from the focus point. As may be appreciated, layersthat are closer to the focus point will have a smaller relativedisplacement value than layers further away from the focus point. Ineach frame of the sequence, each layer's relative displacement ispreferably a multiple of the frame number in the frame sequence. Forexample, in one embodiment, a layer that has a displacement of 2 pixelsrelative to the focus point is displaced 2 pixels in frame 1, 4 pixelsin frame 2, 6 pixels in frame 3 and so forth. This displacement relativeto the focus point can be referred to as a “relative displacement,”since it provides a basis displacement value which is multiplied by aframe number to arrive at a total displacement for a particular layer.As may be appreciated, the object volume is defined by the relativedisplacement value for each layer since the layer displacement magnitudeprovides the illusion of volume. Accordingly, the apparent distancebetween layers can be increased or decreased by changing the relativedisplacement values to thereby stretch or shrink the object. This allowsthe designer great freedom in defining objects with various volumetriccharacteristics.

In one embodiment, the total pixel displacement for all objects islimited to 450 pixels for positive movement (left to right) and 140pixels for negative movement (right to left). The animation displacementpreferably starts from a first frame that provides the object layers ata first extreme displacement value and ends on a final frame thatprovides the object layers at another extreme displacement value. Thefirst frame position preferably includes layers displaced to an extremeleft position with a maximum possible displacement of −450 pixels forpositive relative displacement or +140 pixels for negative displacement.Each layer's relative displacement is inverted and multiplied by half ofthe total number of frames in the animation to arrive at the startingframe. The animation then proceeds to a final frame having a maximumpossible displacement of +450 pixels for positive relative displacementand −140 pixels for negative relative displacement. Thus, by startingthe animation from an already offset maximum position as opposed to theneutral position, the animation range is doubled to include twice thenumber of pixels for movement in both the positive and negativedirections. Hence the animation will start from the offset maximumposition and end at an offset calculated by multiplying each layerrelative displacement by the total number of frames in the animation.Moreover, the animation sequence is fluid with smooth transition from afirst angle to a final angle of view.

The relative displacement of the furthest layer from the focus point canbe expressed as (Total displacement) divided by (Number of frames).Thus, for a symmetrical 32 frame sequence, simulating 3-D viewing, themaximum relative displacement is 280/32 or −8.75 pixels and 900/32 or+28.125 pixels. Hence the layer furthest from the focus point is limitedto a relative displacement value of either −8.75 or +28.125, dependingon the direction of movement. Thus, for layers selected uniformlydistanced from the focus point, each layer's relative displacement canbe expressed as (Maximum relative displacement) divided by (Number oflayers from focus point) multiplied by (Particular layer number fromfocus point). Thus, for the above illustrated simulation having 21layers, the focus point assumed to be in the center layer 11, therelative displacement is −0.875 ((280/32/10)*1) and +2.8125((900/32/10)*1) for layers 10 and 12, −1.75 ((140/16/10)*2) and +5.625((450/16/10)*2) for layers 9 and 13, and so forth. As may beappreciated, fractional displacement values can be associated withlayers since the actual displacement results from multiplying by afactor, which provides a result that is later rounded to the nearestwhole pixel.

The displacement of layers in each frame is preferably the relativedisplacement for each layer multiplied by the frame number from thefocus point. Preferably, the resultant pixel displacement for each layeris rounded to the nearest whole pixel with half values rounded down. Theprocess iterates over the entire frame sequence to provide an animationincluding layers of the object at various horizontal offset positions.In some embodiments, optional simulation effects are selected, such ascolor shift, rotation, opacity, and other filters usable in graphicsoftware to manipulate appearance. Other optional effects includevertical displacement and vertical stretching. The animation frames arethen rendered in accordance with the selected parameters.

FIG. 6 illustrates a sequence of animation frames for the objectillustrated in FIG. 5A. The illustrated sequence includes 9 framesmoving an observation point from left to right. As may be appreciated,in frame 1 more of the right portion of the object is visible to theobserver since the observation point is to the left of the object. Frame1 includes the object layers at a first extreme orientation. Conversely,in frame 9, more of the left portion of the object is visible to theobserver since the observation point has been moved to the right of theobject. Frame 9 includes the object layers and a second extremeorientation. Preferably, the greater the number of layers, which aredefined for an object, the better the animation in simulating volume.The individual frames in the sequence of FIG. 6 are preferably combinedinto an interlaced image as discussed above with reference to generallenticular imaging techniques or are provided to a video motion playersoftware.

Although the present invention was discussed in terms of certainpreferred embodiments, the invention is not limited to such embodiments.A person of ordinary skill in the art will appreciate that numerousvariations and combinations of the features set forth above can beutilized without departing from the present invention as set forth inthe claims. Thus, the scope of the invention should not be limited bythe preceding description but should be ascertained by reference toclaims that follow.

1. A method for producing an input image for application from a twodimensional image onto an image surface adapted to present interlacedframes of the input image, comprising: receiving an input image, theimage including at least one object; receiving layer definitions for aplurality of layers for at least one object of the input image, thelayer definitions include data for object layers that are copied fromthe object by defining an area containing coplanar portions of theobject; receiving an indication of the total displacement for the objectanimation; determining a relative displacement for each layer from theplurality of layers; generating a sequence of image frames byprogressively offsetting said plurality of layers in accordance withrelative layer displacement and a total animation displacement; andinterlacing the animated sequence of frames to a combined image toprovide an input interlaced image for application to said image surface.2. The method of claim 1, wherein said progressive offsetting is byhorizontally offsetting each layer of the plurality of layers.
 3. Themethod of claim 1, further comprising selecting a focus point for theobject, wherein said progressive offsetting for each layer is dependenton the layer location relative to the focus point.
 4. The method ofclaim 3, wherein the focus point is within the object.
 5. The method ofclaim 1, wherein said progressive offsetting includes offsetting in twodirections.
 6. The method of claim 1, wherein the animation offsettingis symmetrical about a center axis of the object.
 7. The method of claim1, wherein the layer definitions include data for object layers that arecopied from the object by defining an area containing coplanar portionsand removing a previously defined portion from the currently definedlayer.
 8. The method of claim 1, wherein said progressive offsetting togenerate a frame sequence, which includes a total frame number,comprises: selecting a focus point, the focus point associated with alayer from the plurality of layers which remains static when anobservation point for the animation is varied; assigning a relativedisplacement for each layer in the plurality of layers by reference tothe received total displacement for the object, the number of frames inthe frame sequence, and the number of layers between the layer and thefocus point; setting the layer displacement to a first extremeorientation by displacing each layer in the reverse to the relativedisplacement direction and by multiplying the reverse relativedisplacement by half of the total frame number; and displacing eachlayer in accordance with the assigned relative displacement and framenumber in the sequence, the displacement of each layer in each frameprovided by multiplying the frame number by the relative displacement toarrive at the layer displacement at a second extreme orientation toprovide a symmetrical sequence of animation frames for the object. 9.The Method of claim 8, wherein the total number of frames is 20.